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Abstract. 

This  paper  describes  a machine  model  Intended  to  be  useful  in 
deriving  realistic  complexity  bounds  for  tasks  requiring  list  processing. 

As  an  example  of  the  use  of  the  model,  the  paper  shows  that  any  such  machine 
requires  non-linear  time  in  the  worst  case  to  compute  unions  of  disjoint 
sets  on-line.  All  set  union  algorithms  known  to  me  are  instances 
of  the  model  and  are  thus  subject  to  the  derived  bound.  One  of  the  known 
algorithms  achieves  the  bound  to  within  a constant  factor. 
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Introduction. 

Computer  scientists  have  attempted  for  many  years  to  derive  lower 
bounds  on  the  complexity  of  computational  problems.  This  effort  has  met 
with  some  success,  providing,  for  example,  exponential  lower  bounds  on  the 
complexity  of  equivalence  for  regular  expressions  [15 ]>  validity  in 
Presburger  arithmetic  [lU],  and  circularity  in  attribute  grammars  [7]. 

In  addition  to  these  bounds  for  hard  problems,  several  results  for  simpler 
problems  exist,  including  boimds  on  the  number  of  comparisons  required  for 
ordering  problems  [9]^  on  the  number  of  data  accesses  required  for  testing 
properties  of  graphs  [15 ]j  and  on  the  number  of  arithmetic  operations 
reqiiired  for  evaluating  various  polynomials  [2]. 

In  spite  of  this  progress,  one  domain,  that  of  list  processing  problems, 
is  almost  entirely  devoid  of  lower  bound  results.  The  subject  of  data 
structures  is  now  part  of  the  standard  computer  science  curriculum,  and 
every  comjiuter  science  library  contains  many  books  on  the  subject.  Yet, 
with  the  exception  of  a few  resiilts  on  the  relative  power  of  various  data 
structures,  nothing  is  known  about  the  inherent  power  of  pointer  manipulation. 

Qie  reason  for  this  state  of  affairs  is  the  lack  of  a thoroughly 
understood  machine  model  wiiich  is  both  realistic  and  theoretically  accessible. 

One  candidate,  the  random  access  machine  [1],  which  has  been  used  by 

several  authors  to  provide  realistic  measures  of  the  complexity  of  various 

algorithms,  seems  too  powerful  to  analyze  easily.  It  also  has  certain 

defects,  such  as  allowing  unbounded  parallelism  if  a "uniform  cost"  measure 

[1]  is  used.  i 

However,  another  possible  model  exists.  In  1955  Kolmogorov  [11,12]  f 

proposed  a machine  which  operates  by  manipulating  pointers  connecting  nodes. 

Fifteen  years  later  Khuth  [8]  proj'osed  a similar  machine,  which  he  called 
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a linking  automaton.  Later  and  independently  Sch^inhage  [l6]  defined  such 
a machine,  which  he  called  a storage  manipulation  machine,  and  showed  that 
such  machines  can  simulate  Turing  machines  with  multidimensional  tapes  in 
real  time.  Althou^  these  machines  provide  a useful  tool  for  describing 
pointer  manipulation  algorithms,  no  bounds  on  their  computational  power 
except  Schttnhage's  seem  to  exist. 

This  paper  describes  an  extension  of  Khuth' s machine,  called  a 
reference  machine.  The  paper  examines  the  ability  of  such  a machine  to 
solve  a problem  requiring  manipulation  of  disjoint  sets,  and  proves  that 
any  reference  machine  which  solves  the  disjoint  set  problem  requires 
non-linear  time  (in  the  worst  case)  to  do  so,  under  certain  natural 
restrictions.  The  lower  bound  is  tight  to  within  a constant  factor. 

This  result  shows  that  it  is  possible  (in  at  least  one  case)  to  derive 
a non-linear  lower  bound  on  the  complexity  of  a list-processing  problem 
using  a realistic  computer  model.  The  resiilt  also  provides  a partial 
solution  to  Kiuth's  exercise  2.6.1  [8]  which  asks  us  to  "Explore  the 
properties  of  linking  automata...". 


r— ______ 


2.  Reference  Machines.  ^ 

A reference  machine  consists  of  a memory  and  a finite  number  of 
registers.  The  registers  are  of  two  types:  data  registers  and  reference 
registers.  The  memory  consists  of  a finite  but  expandable  pool  of  records. 

Each  record  consists  of  a finite  number  of  itemsy  each  of  which  is  either 
a data  item  or  a reference  item.  Each  item  has  an  identifying  name.  All 
records  are  identical  in  structure;  that  is,  they  contain  the  same  items. 

A reference  machine  manipulates  data  and  references.  A reference 
either  specifies  a particular  record  or  is  null  {p)  . Each  reference 
register  and  reference  item  can  store  one  reference.  Data  cein  be  ol'  any 
kind  whatsoever  (integers,  logical  values,  strings,  real  numbers,  vectors, 
etc.).  Each  data  register  and  data  item  can  store  one  datum. 

A program  for  a register  machine  consists  of  a sequence  of  instruct! ons, 
numbered  consecutively  trom  one.  Each  instruction  is  of  one  of  the  following 
eight  types.  (Each  r below  denotes  a reference  register,  each  s denotes 
a data  register,  each  t denotes  a regi ster  of  any  type,  and  each  n 
denotes  an  item  name.) 

r - 0 ITace  a null  reference  in  register  r . 
t^  »-  tg  (t^  and  tg  must  be  of  the  same  typ;e'. 

place  the  contents  of  register  t^  in  register  t^  , erasing 
what  was  there  previously, 
t — n(r)  (n  and  t must  be  of  the  same  type). 

Let  N be  the  n item  of  the  record  specified  by  the  contents 
of  r . place  the  contents  of  N in  register  t , erasing  what 
was  there  previously.  (if  r contains  0 , this  instruction 
does  nothing. ) 
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n(r)  •-  t (n  and  t must  be  of  the  same  type) 


Let  N be  the  n item  of  the  record  specified  by  the  contents 
of  r . Place  the  contents  of  t in  item  N > erasing  what 
was  there  previously.  (if  r contains  , this  instruction 
does  nothing. ) 

Sf  <-  Sg  9 Sj  Combine  the  data  in  registers  s^  and  s^  by  applying  the 
operation  0 . Store  the  res\ilt  in  s^  , erasing  what  was 
there  pre’/iousi.y. 

create  r Create  a new  record  (not  specified  by  any  existing  reference) 
and  place  a reference  to  it  in  r . 
halt  Cease  execution, 

if  condition  then  go  to  i 

If  the  condition  is  true,  then  transfer  control  to  instruction  i 
If  the  condition  is  false,  do  nothing. 


Each  condition  in  an  if  instruction  is  of  one  of  the  following  types. 


Always  true. 


t^  = tg  (t^  and  tg  must  be  of  the  same  type) 

True  if  the  contents  of  t^  and  t^  are  the  same. 


p(s^, Sg)  True  if  the  contents  of  s^  and  s^  satisfy  the  predicate  p , 
wb'ere  p is  any  predicate  on  data. 
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A reference  machine  executes  a program  instruction-by-inrtruction 


in  consecutive  order,  beginning  with  instruction  one.  Execution  of  an 
if  instruction  may  cause  control  to  be  transferred  to  a non-consecutive 
instruction,  in  which  case  consecutive  execution  resumes  from  this  new 
instruction.  When  the  machine  reaches  a halt  instruction,  execution 
ceases.  The  last  instruction  of  every  program  is  a halt  . 

A reference  machine  step  consists  of  the  execution  of  a single 
instruction.  The  running  time  of  a reference  machine  prograjn  is  the 
n\jmber  of  steps  the  machine  requires  to  execute  the  program,  as  a 
f\mction  of  the  initial  state  of  the  registers  and  memory.  The  storage 
space  required  by  a reference  machine  program  is  the  number  of  records 
initially  in  memory  plus  the  number  created  during  execution. 

When  a new  record  is  created  all  its  items  initially  contain  a 
special  value  ctalled  \mde fined  (a)  . The  initial  value  of  any  register 
may  also  be  A . If  a reference  machine  attempts  to  use  the  contents  of 
a register  or  item  containing  A > it  halts.  However,  the  machine  is 
allowed  to  store  another  value  into  a register  or  item  containing  A • 

I shall  be  uninterested  in  constant  factors  in  running  time  and 
storage  space.  With  this  assumption,  the  register-to-register  assignment 
is  a redundant  Instruction  type  since  it  can  be  simulated  by  a create  , 
a register- to-memory  assignment,  and  a memory-to-reglster  assignment. 
Similarly  uses  of  the  null  reference  value  can  be  deleted  without  affecting 
running  time  by  more  than  a constant  factor.  Extending  the  machine  model 
by  allowing  several  tyq)es  of  records  has  the  effect  only  of  saving  a constant 
factor  in  storage  space. 


To  completely  specil'y  a register  machine,  one  must  describe  the  data 
and  the  types  of  operations  allowed  on  the  data,  iinuth's  linking  automaton 
is  a register  machine  wiiose  data  consists  of  symbols  selected  from  some  set. 

No  operations  on  data  are  allowed  except  testing  for  equality.  Henceforth 
we  shall  use  the  term  symbol  in  a technical  sense  to  refer  to  data  on 
which  no  operations  are  permitted  except  testing  for  equality. 

A pure  reference  machine  is  a register  machine  with  data.  It  is 
not  hard  to  show  that  any  linking  automaton  with  a finite  set  of  symbols 
can  be  simulated  by  a pure  reference  machine  with  a loss  of  only  a constant 
factor  in  running  time.  I shall  consider  examples  of  reference  machines  which 
have  integers  as  data  and  addition  and  comparison  as  allowed  operations. 

The  lower  bound  result  holds  for  all  reference  machines,  whatever  their  data. 

In  a reference  machine,  access  to  memory  is  by  explicit  reference  only; 
no  computation  on  references  is  possible.  The  reference  machine  model  is 
thus  apparently  less  powerful  than  the  random  access  model  with  uniform  cost 
measure  [1];  reference  machines  lack  the  ability  to  use  address  arithmetic 
for  such  purposes  as  manipulating  a hash  table  [9],  performing  a radix 
sort  [ 9 ]>  or  accessing  a dense  matrix  [ 8 ].  These  machines  are,  however, 
powerful  enough  to  simulate  such  list-processing  languages  as  LISP  and  to 
model  the  list-processing  featiores  of  Algol-W,  I'L/1,  and  other  general 
purpose  languages. 

It  wouJ-d  of  course  be  possible  to  study  the  general  properties  of 
reference  machines,  comparing  their  power  with  that  of  other  classes  of 
automata,  as  Schftnhage  [16]  has  done.  Here,  however,  I analyze  the  ability 
of  reference  machines  to  solve  a specific  problem  in  list  processing. 


3. 


The  Disjoint  Set  Union  problem. 


Let  Sg, be  n disjoint  sets,  each  containing  a single 
element.  The  disjoint  set  union  problem  is  to  carry  out  a sequence  of 
operations  of  the  following  two  types  on  the  sets. 

find(x)  ; determine  the  name  of  the  set  containing  element  x . 

union (A, B)  : add  all  elements  of  set  B to  set  A (destroying 

set  B ). 

The  operations  are  to  be  carried  out  on-line;  that  is,  each  instruction 
must  be  conpleted  before  tlie  next  one  is  known.  V/e  shall  ajsur.e  that  the 
sequence  of  operations  contains  exactly  n-1  union  operations  (so  that 
after  the  last  \xnion  all  elements  are  in  one  set)  and  m > n intermixed 
find  operations  (if  m < n , some  elements  are  never  found). 

The  disjoint  set  union  problem  is  an  abstraction  of  the  op-erations 
necessary  to  implement  FORTRAIi  EQUTVALEITCE  and  COtiMON  statements  [ 5 ]. 
Algorithms  for  this  problem  and  for  a generalization  of  it  have 
applications  in  graph  theory  [l8],  global  code  optimization  [18,19],  and 
linear  algebra  [19l.  A number  of  algorithms  exist  [1,1, 5, 6], 

A reference  machine  solution  to  the  set  union  problem  consists  of  a 
reference  machine,  a representation  of  the  input  sets  as  collections  of 
records,  a program  for  carrying  out  a find  , and  a program  for  carrying 
out  a union  . The  reference  machine  solves  the  set  union  problem  in  the 
following  way.  Initially  the  machine  memory  represents  the  input  sets. 
r^Lch  find  is  carried  out  by  executing  the  find  program,  which  halts 
having  identified  the  set  containing  the  desired  element.  Each  union 
is  carried  out  by  executing  the  union  pirogram,  which  lialts  having 
modified  the  contents  of  memory  to  reflect  the  union.  I shall  make  the 


fullowlng  assumptions  concerning  the  details  of  this  process. 


(5.1)  Each  set  and  each  element  has  a distinct  associated  symbol. 

(3.2)  No  record  in  the  collection  for  an  input  set  contains  the  symbol 

of  any  other  set  or  of  any  element  outside  the  set. 

(3.3)  No  record  in  the  collection  for  an  input  set  contains  a reference 

to  any  record  outside  the  collection. 

(3.^+)  Before  the  find  program  is  executed  to  locate  the  set  containing 
an  element  x , a reference  to  some  record  containing  the  symbol 
for  X is  placed  in  the  designated  input  register  r^  and  A is 
placed  in  all  other  registers.  The  find  program  halts  with  the 
symbol  for  the  set  containing  x in  the  designated  output  register  s^ 

(5.5)  Before  the  union  program  is  executed  to  add  elements  in  set  B to 

set  A , references  to  records  containing  the  symbols  for  A and  B 

are  placed  in  the  designated  input  registers  r^  and  r^  , respectively 
and  A is  placed  in  all  other  registers.  The  union  program  halts  with 
no  output. 

The  sequence  of  steps  associated  with  a set  union 
problem  and  a reference  machine  solution  is  the  sequence  of  steps 

executed  by  the  machine  when  it  carries  out  the  finds  and  unions  . The 

length  of  this  sequence  measures  the  total  running  time  of  the  machine. 

The  main  result  of  this  paper  is  a non-linear  lower  bound  (as  a function 
of  n and  m ) on  the  length  of  any  sequence  of  steps  which  solves  a 
worst-case  instance  of  the  set  union  problem. 

The  formulation  described  above  is  intended  to  be  realistic  and  to 
facilitate  derivation  of  a lower  bound.  Assumption  (3.I)  above^  requiring 
that  sets  and  elements  be  represented  by  symbols,  makes  it  impossible  to 
encode  all  elements  of  a set  into  a single  datum  and  to  move  this  datum  at 
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a cost  of  one  step  per  move;  vd.thout  this  restriction  there  is  a rel'erence  machine 
which  can  solve  any  set  union  problem  in  linear  time.  Assumptions  (5.2),  (3»5)> 
and  (3-^)  iinply  that  the  machine,  wiien  performing  a find  on  some  element  x , 
has  access  only  to  records  representing  the  set  containing  x . Assumptions 

(3.2) ,  (3"3)j  and  (3.5)  imply  that  the  isachine,  when  performing  a union  on 
sets  A and  B , has  access  only  to  records  representing  the  sets  A and  B . 

It  follows  by  induction  on  the  number  of  finds  and  unions  that  (5.2')  and 

(3.3)  hold  for  the  sets  existing  at  any  time  during  the  computation,  not  just 
for  the  input  sets.  In  other  words,  the  contents  of  memory  after  any 

parti cixlar  find  or  union  can  be  partitioned  into  collections  of  records 
such  that  each  collection  corresponds  to  a ciirrently  existing  set,  all 
symbols  for  the  set  and  its  elements  occur  only  in  the  corresponding 
collection  of  records,  and  no  record  in  one  collection  contains  a reference 
to  a record  in  another  collection.  Without  assumptions  (3. 2)- (3. 5)  any 
particular  instance  of  the  set  union  problem  can  be  solved  in  linear  time 
by  initially  moving  symbols  for  all  sets  and  elements  into  a single  record 
and  solving  all  finds  by  accessing  only  this  record,  thou^  I conjecture 
that  even  without  assumptions  (3.2)-(3.5)  no  single  reference  machine  can 
solve  a.1 1 instances  of  the  set  union  problem  in  linear  time. 

If  an  algorithm  for  the  set  union  problem  is  to  be  useful  in  practice, 
the  symbol  of  each  set  and  of  each  element  should  be  scored  in  exactly  one 
record,  so  that  the  initijilization  for  finds  (5«'^)  a^d  unions  (3.5)  is 
uniquely  defined.  All  tVie  algorithms  to  be  considered  have  this  property, 
but  the  lower  bound  proof  does  not  require  it. 
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U.  Algorithms  for  the  Set  Union  Problem. 


All  algorithms  for  the  set  union  problem  knovm  to  me  can  be  implemented 
on  reference  machines.  This  section  describes  six  such  algorithms.  These 
algorithms  are  of  two  general  types,  quick  find  , requiring  constant  time 
for  each  find  , and  quick  union  , requiring  constant  time  for  each 
union  . All  the  algorithms  represent  each  input  set  by  a single  record, 
containing  the  symbol  for  the  corresponding  set  and  the  symbol  for  the 
corresponding  element  in  data  items  set  and  element  , respectively. 

Each  element  is  peimiarjently  associated  with  the  record  containing  its 
symbol,  and  no  new  records  are  ever  created.  During  the  computation, 
a currently  existing  set  is  represented  by  the  collection  of  records 
corresj^onding  to  its  elements  and  the  sjmibol  for  the  set  is  contained  in 
exactly  one  of  these  records. 

In  the  quick  find  method,  each  record  contains  two  reference  items, 
parent  and  next  . One  record  in  the  collection  representing  a set 
contains  the  symbol  of  the  set.  The  qarents  of  all  records  in  the 
collection  refer  to  this  header  record.  The  next  items  link  all  records 
in  the  collection  into  a list  whose  first  element  is  the  header.  Figure  1.1 
illustrates  this  data  structure. 

[Figure  1.1] 

With  this  representation,  a find  requires  two  reference  machine 
steps;  one  to  access  the  q airent  of  the  input  record  (which  refers  to  the 
header)  and  one  to  access  the  set  of  the  header.  A union  of  A and  B 
requires  seven  steps  per  element  in  B ; each  record  in  the  collection  for 
B must  have  its  parent  modified  to  refer  to  the  header  of  A and  must 
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be  linked  into  the  list  for  A . Table  i*.l  contains  programs  in  Algol-like 
notation  for  union  and  find  . It  is  easy  to  translate  these  into 
reference  machine  programs. 

[Table  1.1] 

Adding  a heuristic  to  the  union  program  improves  its  performance 
considerably.  Each  record  needs  an  additional  data  item,  size  . The 
size  item  is  only  meaningful  for  headers;  it  coiints  the  number  of  elements 
in  the  corresponding  set.  To  perl'orin  a union  of  A and  B , the  size  of 
A is  compared  to  the  size  of  B . If  B is  smaller,  the  union  proceeds 
as  before.  If  A is  smaller,  the  symbols  for  A and  B in  the  headers  of 
the  sets  are  interchanged,  the  references  in  r^  and  r^  to  the  headers 
are  interchanged,  and  the  union  proceeds  as  before.  The  time  required  for 
such  a weighted  union  is  proportional  to  the  size  of  the  smaller  ol’  A 
and  B . Table  1.2  contains  a program  for  this  heuristic. 

[Table  1.2] 

In  the  quick  union  method,  each  record  contains  only  one  reference 

item,  parent  . The  collection  of  records  representing  a set  forms  a rooted 

/ 

tree-^  with  the  parent  of  each  record  referring  to  its  parent  in  the  tree; 


— ' A rooted  tree  T is  a connected,  acyclic,  undirected  graph  with  a 
unique  distinguished  vertex  r , called  the  root  of  T . If  v and 
w are  vertices  of  T such  that  v is  on  the  (unique)  simple  path 
from  r to  w , then  v is  an  ancestor  of  w and  w is  a descendant 

of  V , This  relationship  is  denoted  by  v w . The  relationship 

V -*  w and  v ^ w is  denoted  by  v -»  w . If  v -»  w and  (v,  w)  is 
an  edge  of  T , then  v is  the  parent  of  w and  w is  a child  of  v . 
This  relationship  is  denoted  by  v -♦  w . A leaf  is  a vertex  with  no 
children.  The  height  of  a vertex  v is  the  length  (number  of  edges) 
of  the  longest  simple  path  from  v to  a descendant  of  v . The 

subtree  of  T rooted  at  vertex  v is  the  subgraph  of  T induced  by 

the  descendants  of  v , with  v as  root. 
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the  parent  of  the  root  is  . The  root  contains  the  symbol  of  the  set. 

Figure  h.2  illustrates  this  data  structure, 

[Figure  1+.2] 

With  this  representation,  a lanion  of  A and  B requires  only  one 
machine  step,  to  place  a reference  to  the  root  of  A in  the  parent  of 
the  root  of  B . A find  is  performed  by  starting  from  the  input  record 
and  follovdng  parent  references  until  reaching  a record  with  a null 
parent  ; this  record  is  the  root  of  the  tree  representing  the  set  and 
contains  the  set  symbol.  The  find  requires  time  proportional  to  the 
number  of  records  on  the  path  from  the  input  record  to  the  root.  Table  I.5 
contains  programs  for  these  versions  of  union  and  find  . 

[Table  h.3] 

The  weighted  xinion  heuristic  can  be  added  to  quick  union  ; it  uses 
extra  time  on  unions  but  may  save  time  on  later  finds  . A heuristic 
for  finds  called  path  con^^ressjon  is  also  useful.  After  a find  , every 
record  on  the  path  from  the  input  record  to  the  root  has  its  parent 
modified  to  refer  directly  to  the  root.  Path  compression  increases  the 
running  time  of  a find  by  a constant  factor  but  may  save  time  on  later 
finds  , Table  t.U  contains  programs  for  union  and  find  with  these 
heuristics. 

[Table  U.U] 

The  quick  find  algorithms  are  apparently  part  of  the  folklore  of 
compiler  construction;  a description  of  these  algorithm.s  appears  in  [1].  The 
quick  union  algorithm  with  the  weighted  union  heuristic  was  first  presented 
in  [5].  The  path  compression  heuristic  is  apparently  due  to  Mcllroy  and 
Morris  [1].  Worst-case  analysis  of  these  algorithms  appears  in  [1,1,5,6,17] ; 

Table  1.5  summarizes  the  results.  The  theoretically  best  algorithm  in  the 
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worst  case  is  quick  union  with  both  heuristics;  its  mnning  time  is 
0(m  a(m, n))  , where  a:(m, n)  is  a functional  inverse  of  Ackermann’s 
!\inction  defined  as  follows. 

For  i,  j > 0 let  the  function  A(i,J)  be  defined  by 

(li.l)  A(i,0)  = 0 ; 

A(0,j)  = 2-^  for  j > 1 ; 

A(i,l)  = A(i-1,2)  for  i > 1 ; 

A(i,J)  = A(i-1 , A(i,  j-1))  for  i > 1 , j > 2 . 

Let 

(U.2)  a(i,n)  = min{jlA(i,  j)  ^ log^  n} 

and 

* / 

(1.3)  a(m,n)  = inin[i  > 1 | A(i,  [_2m/nj  ) > log^  n]  .-I 

[Table  k.5l 

Yao  [21],  Doyle  and  Rivest  [5],  and  Khuth  and  Schttnhage  [10]  have 
carried  out  average-time  analyses  of  the  algorithms  for  several  reasonable 
probability  measures  under  the  assumption  that  m and  n are  proportional. 
Table  h.6  contains  the  results  of  Yao  and  Khuth  and  Schttnhage  for  one 
measure  (see  [21]). 

[Table  h.6] 

The  quick  union  algorithm  is  simpler  and  requires  less  storage  than 

the  quick  find  algorithm  and  is  thus  more  useful  in  practice.  Whether 

either  of  the  two  heuristics  should  be  used  with  this  algorithm  depends  upon 

the  size  of  the  problem  and  the  cost  of  time  versus  the  cost  of  space.  The 

average  running  time  of  the  quick  union  algorithm  with  path  compression 

but  without  wei^ted  union  is  ixnknown  for  the  probability  measure  used  by 

Yao  and  Khuth  and  SchBnhage. 

__ 

— ' For  any  real  number  x , |_xj  denotes  the  greatest  integer  not  larger 
than  X . 

li 
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r 

When  path  comjression  is  used,  the  running  time  of  the  quick  union 

algorithm  tends  raj  idly  to  0(m)  as  m/n  increases.  For  instance,  if 

weighted  union  is  not  used  -md  m/n  _•  cn^^^  for  some  positive  constants 

c and  • , the  running  time  is  0(m)  . If  weighted  union  is  used  and 

m'n  ca(k,  n for  some  jiositive  constants  c and  k,  the  running  time  is 

0(m)  . Note  that  a(0,n  is  O(log  log  n)  and  a(l,n)  is  O(log  n)  , where 

i times 

^ ^ 

log  n = min{i  ( log  log  ...  log  n < 1}  . 

The  weighted  union  rule  requires  that  records  contain  integer  data 
items  and  that  reference  machines  add  and  compare.  It  is  natural  to  ask 
whether  the  weighted  union  rule  can  be  implemented  on  a pure  reference 
machine  in  such  a way  that  the  total  time  for  all  unions  is  O(n')  . 

The  answer  is  yes. 

Each  non-negative  integer  is  represented  by  a list  which  encodes  the 
binary  digits  of  the  integer.  A zero  is  encoded  by  a null  pointer;  a one 
is  encoded  by  a non-null  pointer.  The  digit  list  is  singly  linked  from 
the  low  order  digit  to  the  hi^  order  digit.  Figure  U.5  illustrates  this 
representation. 

[Figure  k.5] 

Two  integers  are  added  by  scanning  the  digit  lists  and  adding  digit- 
by-digit,  propagating  carries  in  the  usual  fashion.  The  scan  stops  after 
the  end  of  the  shorter  list  is  reached  and  the  last  carry  stops  propagating. 
Two  integers  are  compared  by  scanning  both  simultaneously  and  noting  the 
highest  order  digit  on  which  they  differ.  The  scan  need  only  extend  to  the 
end  of  the  shorter  digit  list;  the  integer  with  the  longer  digit  list  must 
be  larger.  I leave  as  an  exercise  the  implementation  of  these  algorithms 
as  register  machine  programs. 

The  n-1  union  operations  carried  out  by  the  quick  union  method 
perform  the  following  arithmetic.  Initially  there  are  n integers,  each 
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equal  to  one.  During  a union  , two  of  the  integers  are  coinjared  and 
then  added.  After  n-1  unions  , a single  integer  equal  to  n remains. 
Since  comparing  two  integers  requires  no  more  time  than  adding  them, 
it  will  suffice  to  bound  the  time  required  by  all  the  additions. 


Lemma  4.1.  Let  a,  b,  c be  integers  such  that  a+b  = c and  let  (a^^) 

00 

(b.)  , (c..  ) , respectively,  be  their  binary  digit  lists  ( a = T,  a.2 

i=  0 

CO  . 00  . 

b = 12  b.2^  , c = S c.2^  ; a.,b.,c.  e {0,1}  ).  Let  d.  be  the 

i=0^  i=0^  111  1 

carry  from  the  i-th  position  when  a and  b axe  added.  Then 
k k 

^ (a. +b.)  = cl  + 22  (c.+d.)  for  all  k . In  particular, 

i=0  ^ ^ ^ i=0  ^ ^ 

00  00 


22  (a.+b.)  = 22  (c.+d.)  . 

• ^ 1 1 • r.  1 1 

1=0  1=0 


Proof.  For  i > 0 , 1 ^x  ^ ^*^1  (^-ssuming  p = 0 ). 

Thus  ^x  ^ ^i  " ^i  *^i  ^ ^'^i  " '^i  1^  * Summing  from  i = 0 to  i = k 


gives  the  lemma. 


U 


The  time  needed  to  add  two  binary  integers  by  reference  machine 


’I 


is  proportional  to  the  length  of  the  shorter  integer  plus  the 
number  of  carries.  By  Lemma  1.1,  the  total  number  of  ones  in  the 
binary  representations  of  both  integers  is  equal  to  the  number  of  ones  in 
the  binary  representation  of  the  sum  plus  the  number  of  carries. 

Consider  the  arithmetic  performed  during  the  union  operations. 

Initially,  the  total  number  of  ones  in  the  binary  representations  of  all 
the  set  sizes  is  n . Each  carry  performed  during  an  addition  causes  the 
total  nmber  of  ones  to  decrease  by  one.  Thus  the  total  number  of  carries 
cannot  exceed  n-1  , and  the  time  required  for  all  carries  is  0(n)  . 

It  remains  to  bound  the  total  length  of  the  shorter  of  each  pair  of 
integers  added  during  union  operations.  Let  f(n)  be  a worst-case  bound 
on  this  total  length  as  a function  of  n . "^hen  f(l)  = 0 , and 

f(n)  = maxfLlog^  kj  + 1+  f(k)  + f(n-k)  j 1 5 h < n/2}  for  n > 1 , 

since  the  length  of  the  binary  representation  of  k is  Ll°S2  • 

Lemma  1«2.  f(n)  < Jn  - 2 log2(n+l)-l  . 

Proof.  By  induction  on  n . 

f(l)  = 0 < 3 - 2 log^  2 - 1 . 
f(2)  = 1 < 6-2  log^  3 -1  . 

Let  n > 3 and  suppose  the  lemma  is  true  for  all  values  less  than  n . 
Let  k be  such  that  1 < k < n/2  and 

f(n)  = Llog^  kj  + 1 + f (k)  + f (n-k)  . 

By  the  induction  hypothesis 
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f(n)  < logg  k + l+5k-2  log^Ck+l)  - 1+ 5 (n-k)  - 2 logg(n-k+l)  - 1 
5n  - 1 - loggCk+l)  - 2 logg(n-k+l)  . 

The  function  -log^Ck+l)  -2  log^Cn-k+l)  for  1 5 h < n/2  is  maximum 
viien  k = 1 . Thus 

f (n)  < 3n  - 1 - log^  2-2  log^  n 

< 3n  - 2 - 2 logg  n 

Also,  n > 3 implies  ^2  n > n+1  , vdiich  means 

-2  log2(n+l)  > -2  log^'/i^n  = -2  log^  n-1  , 

and 

f(n)  < 3n  - 2 logg(n+l)-l  . □ 

It  follows  that  the  total  time  to  perform  all  arithmetic  associated 
with  the  union  operations  is  0(n)  , and  the  following  theorem  holds. 

Theorem  4,1.  There  exists  a pure  reference  machine  which  solves  any  disjoint 
set  union  problem  in  0(m  a(m,n))  time. 
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5 . A Non-Lineax  Lower  Bound. 


This  section  shows  that  for  all  m and  n there  is  a set  union 
problem  which  requires  at  least  cma(m, n)  steps  to  solve  by  reference 
machine,  where  c is  a positive  constant  independent  of  m and  n . 

Rather  than  consider  reference  machines,  I consider  sequences  of  reference 
machine  steps.  Given  a set  union  problem,  a sequence  of  reference  machine 
steps  is  said  to  solve  it  if  there  is  some  reference  machine,  some  set  of 
union  programs,  one  for  each  union,  and  some  set  of  find  programs,  one 
for  each  find,  such  that  when  the  sequence  of  programs  corresponding  to 
the  sequence  of  union  and  find  operations  is  executed  according  to  the 
conventions  of  Section  5,  the  given  sequence  of  reference  machine  steps 
results  and  the  find  programs  produce  correct  answers.  Note  that  any 
sequence  of  reference  machine  steps  can  be  carried  out  by  a non-branching 
reference  machine  program.  The  first  step  in  the  lower  bound  proof  is  to 
convert  into  a simple  normal  form  any  sequence  of  reference  machine  steps 
which  solves  a set  union  problem. 

Theorem  ^,1.  Let  be  any  sequence  of  reference  machine  stejs  which 

solves  a set  uriion  problem.  Then  there  is  a sequence  of  reference  machine 
steps  Sg  which  also  solves  the  set  union  problem  and  has  the  following 
properties : 

(5.1)  IS2I  < 2(m+  n+  \g^\). 

(5.2)  S.,  manipulates  no  data  except  set  and  element  symbols. 

(5.3)  Sg  represents  each  input  set  by  a single  record  and  contains  no 
create  instruction. 

(5.^)  Sg  fetches  a symbol  from  memory  only  as  the  last  instruction  of 
a find  . 
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I. 


Proof. 


Let  be  a sequence  of  rei'erence  machine  stej  s which  solves  some 

set  union  problem.  Delete  from  all  steps  which  manipulate  data  other 

than  set  and  element  symbols.  The  sequence  now  satisfies  (5.2)  and 

stiU  solves  the  set  union  problem. 

The  sequence  to  be  constructed  manipulates  records  corresponding 

to  the  sets,  the  elements,  and  the  records  manipulated  by  . Initially 
tiie  memory  of  consists  of  one  record  for  each  input  set  A = [a]  . 

This  record  is  the  representative  of  the  set  A , of  the  element  a , and 
of  each  record  in  the  initial  collection  of  records  by  which 
represents  A . Each  record  created  by  also  has  a representative  in 

the  memory  of  > defined  as  follows.  The  representative  of  a record 
created  during  execution  of  find(a)  is  the  representative  of  a . The 
representative  of  a record  created  during  execution  of  union (A,  B)  is  the 
representative  of  A . For  any  object  x (set,  element,  or  record),  let 
X*  denote  the  representative  of  x . 

Sg  simulates  step-by-step.  If  and  ane  executed  in 

parallel,  the  memory  and  registers  of  Sg  correspond  to  the  memory  and 
registers  of  in  the  following  way. 

(5.5)  If  and  are  records  in  the  memory  of  such  that 

contains  a reference  to  R^  , then  R^  contains  a reference  to 

* » # 

R^  (unless  Rp  = Rp 

(5.6)  If  R is  a record  containing  a set  or  element  symbol  x , then 

» ► » 

R contains  a record  to  x and  x contains  a reference  to 

R*  (unless  R*^  = x"  ). 

(5.7)  If  some  register  of  contains  a reference  to  a record  R , 

then  some  register  of  contains  a reference  to  R 


I 

i 

I 
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(5.8) 


If  some  regiser  of  contains  a set  or  element  symbol  x , 

then  some  register  of  Sg  contains  a reference  to  x*  . 

(5.9)  During  execution  of  find(a)  , Sg  maintains  a reference  to  a 
in  a register.  During  execution  of  union (A,  B)  , Sg  maintains 

■K 

a reference  to  A in  a register. 

Initially  the  memory  of  Sg  consists  of  all  the  representatives,  each 
containing  the  symbol  of  the  corresponding  set,  the  symbol  of  the 
corresponding  element,  and  no  pointers.  Properties  (5.5)  - (5.9)  hold 
initially. 

Let  find(a)  be  a typical  find.  begins  find(a)  with  a reference 

in  r^  to  a record  R containing  the  symbol  for  a . If  (5.6)  holds  before 
the  find,  either  R*  = a*  or  a*  contains  a reference  to  R . Sg  begins 
the  find  with  a reference  to  a*  in  r^  . Sg  ' s first  step  is  to  fetch  a 
reference  to  R*  into  a register.  This  preserves  (5.5)  - (5.9). 

Let  uni on (A,  B)  be  a typical  union.  begins  uni on (A,  B)  with 

references  in  r^  , rg  to  records  R^  , Rg  containing  the  symbols  for 
A , B , respectively.  If  (5.6)  holds  before  the  find,  either  R = A 
or  A contains  a reference  to  R^  ; similarly  either  Rg  = B or  B 

contains  a reference  to  Rg  . Sg  begins  the  union  with  references 

* -X- 

to  A , B in  r^  , rg  , respectively.  Sg  ' s first  two  steps  are  to 

* * 

fetch  references  to  R^  and  Rg  into  registers.  This  preserves 

(5.5)  - (5.9). 

Sg  simulates  each  step  of  S^  in  the  following  way. 

Each  time  S,  fetches  a reference  to  a record  R_  from  a record  R,  , 

X c.  ± 

Sg  fetches  a reference  to  Rg  from  R^  (possible  by  (5.5)).  Each  time 
stores  a reference  to  a record  Rg  in  a record  R^  , Sg  stores  a 
reference  to  Rg  in  R^  (possible  by  (5.7)).  Each 
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time  fetches  a set  or  element  symbol  x from  a record  R , 

fetches  a reference  to  x from  R (possible  by  (5.6)).  Each 
time  stores  a set  or  element  symbol  x into  a record  R , Sg 

stores  a reference  to  x*  in  R and  a reference  to  R in  x (possible 
(5.7)  and  (5.8)).  Each  time  creates  a record,  Sg  does  nothing. 

At  the  end  of  each  find,  Sg  fetcii  ;S  the  appropriate  set  symbol.  Each 
of  these  steps  preserves  (5.5)  - (5.9).  The  sequence  Sg  constructed 
in  this  way  carries  out  the  finds  and  satisfies  (5.1)  - (5.^).  □ 

One  can  represent  the  memory  manipulated  by  a reference  machine  as 
an  undirected  graph,  with  one  vertex  R for  each  record  R and  one  edge 
for  each  reference.  If  a record  R^  contains  a reference  to  a record  Rg  , 
then  (R*, R^)  is  an  edge  in  the  graph.  This  representation  motivates  the 
following  definition,  idilch  reformulates  the  set  union  problem  as  a graph 
construction  problem. 

A link  solution  to  a set  union  problem  consists  of  a set  of  vertices 

V , one  for  each  initial  set  and  element,  and  a sequence  of  instructions 
of  the  fomn  link(v, w)  where  v, weV  . The  sequence  of  link  instructions 
constructs  a graph  edge-by-edge,  starting  from  the  graph  with  vertex  set 

V and  no  edges;  link(x, y)  constructs  edge  (x,y)  . For  any  initial  set 
or  element  x , let  x denote  the  corresponding  vertex.  The  sequence  of 
link  instructions  must  satisfy  the  following  properties. 

(5.10)  The  sequence  of  links  can  be  partitioned  into  contiguous  subsequences, 
each  subsequence  corresponding  to  a union  or  find  operation. 

(5.11)  Let  find(a)  with  answer  A be  a typical  find.  Each  link(x, y) 
in  the  subsequence  for  find(a)  is  such  that  x = A and  the 
distance  between  x and  y in  the  graph  existing  before  the 
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link  is  two.  The  instruction  link (A  , a ) occurs  either  in  the 
subsequence  for  find(a)  or  earlier  in  the  sequence. 

(5.12)  Let  union (a, B)  be  a typical  union.  Each  link(x, y)  in  the 

* 

subsequence  for  uni on (A,  B)  is  such  that  x = A and  either 
y = B or  the  distance  between  x and  y in  the  graph 
existing  before  the  link  is  two. 

Theorem  5.2.  Any  set  union  problem  solvable  in  k reference  machine  steps 
has  a link  solution  of  length  not  exceeding  5ni+^n+kk  . 

Proof.  Let  be  a sequence  of  k reference  machine  steps  which  solves 

a set  union  problem.  Let  be  a sequence  of  reference  machine  stejs 

satisfying  Theorem  5-1.  Then  jSgj  < 2(iir-n+k)  . From  we  construct 

a link  solution  satisfying  the  theorem.  The  vertex  set  for 

consists  of  one  vertex  R for  each  record  R manipulated  by  S„  . If 
Sg  and  S,  are  executed  in  parallel,  the  following  properties  hold. 


(5.13)  If  a record  R^  contains  a reference  to  a record  R^  , then 

•X  X- 

the  distance  between  R^^  and  R^  is  at  most  two. 

(5.1^)  Let  find(a)  with  answer  A be  a typical  find.  If  during  this 
find  some  register  of  contains  a reference  to  R , then 

-M-  ^ -K- 

either  A=R  or  (A,R)  isa  previously  constructed  edge. 

(5.15)  Let  union  (A,  B)  be  a tyj)ical  union.  If  during  this  union 

some  register  of  contains  a reference  to  R , then  either 

^ -X- 

A = R or  (a  ,R  ) is  a previously  constructed  edge. 

simulates  instruction-by- instruction.  Certainly  (5 .13 ) - (5 • 15 ) 

hold  initially.  Let  union  (A,  S')  be  a typical  union.  To  begin  the  union, 
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Let  find (a) 


links  A and  B . This  preserves  (5.13)  - (5.15). 
with  answer  A be  a typical  find.  Suppose  S fetches  I items  from 
memory  while  carrying  out  the  find.  If  (5.15)  holds  before  the  find,  there 

^ 'A 

must  be  a path  of  length  21  or  less  between  A and  a in  the  graph 
existing  before  the  find.  To  begin  the  find  S^  links  each  vertex 
on  this  path  to  A . This  preseirves  (5.13)  - (5.15). 

Consider  a subsequence  of  S2  corresponding  either  to  a find(a) 
with  answer  A or  a union (A,  B)  . Suppose  fetches  a reference 

(say  to  R^)  from  a record  (say  ).  If  (5.13)  - (5.15)  hold  before 
the  fetch,  then  there  is  a path  between  A and  R^  of  length  at  most 
three.  links  each  vertex  on  this  path  to  A . This  preserves 

(5.13)  - (5.15).  All  other  instructions  in  S^  do  not  affect  (5.15)  - (5.15 '1 
in  order  to  store  a reference  (say  R^  ) in  a record  (say  R^^  ),  S^  must 
first  have  references  to  and  R^  in  registers.  By  (5.11)  and  (5.15) 

this  means  that  the  distance  between  R^  and  R^  in  the  graph  existing 
before  the  store  is  at  most  two. 

The  total  length  of  the  sequence  constructed  in  this  way  is 

at  most  5m  + kn  + Ik  , and  the  sequence  clearly  solves  the  set  union 
problem.  □ 

In  the  following  discussion  I shall  not  distinguish  between  an 
initial  set,  its  single  element,  and  the  vertex  representing  the  set  and 
the  element.  Corresponding  to  the  sequence  of  imions  in  any  set  union 
problem  is  a rooted  tree,  called  the  union  tree,  whose  vertices  are  the 


initial  sets  and  whose  edges  are  the  pairs  (A,  B)  such  that  union(A,  B'' 


occTjrs  in  the  sequence.  The  root  of  the  tree  is  the  set  remaining  after 
1 unions  are  carried  out.  With  this  definition,  every  lihk:(v, w)  in 
a link  solution  to  a set  union  problem  has  the  property  that  v -•  w in 
the  union  tree.  In  the  worst-case  set 


union  problems  to  be  constructed 
below,  the  union  tree  is  a complete  binary  tree. 

The  lower  bound  proof  makes  use  of  a rapidly  growing  function  B(i, J 
defined  for  i,.j  > 1 as  follows. 


(5.16)  B(l, j)  =1  for  J > 1 ; 

B(i,l)  = B(i-1,2)+1  for  i > 2 ; 


B(i,j)  = B(i,J-l)  + B(i-1,2®*'^''^"^^)  for  i,j  > 2 . 


Lemma  5.1.  B(i, j)+l  < A(i, 2j)  for  i,  j >1  . 


Proof.  It  is  easy  to  show  by  induction  that  A(i, j)  < min[A(i+l, J ), A(i, j+1) ] 
for  i > 0 , j > 1 . Also, 


(5.17)  A(i,J)  = A(i-l,A(i,  j-1))  = A(i-2,  A(i-1,  A(i,  j-1) ) ) 


2A(i,  j-1)  A(i, i-l)+2 
> 2 > 2 ^ for  i,  j > 2 


The  lemma  follows  by  double  induction  on  i and  j : 


(5.18)  B(1,J)+1  = 2 < 2-^  = A(0,j)  < A(l,2j)  for  j >1  ; 


(5.19)  B(i,l)+1  = B(i-l,2)+2  < A(i-l,l*)+2  < A(i-1,6) 

< A(i-1,  A(2, 1) ) < a(1-1,  A(i,l) ) = A(i,2)  for  i >2  , 
if  B(i-1,2)+1  A(i-2,4)  ; 


1 


(5.20) 


< A(i,2j-2)  + A(i-1,2.2^^^^^'^"^^) 

< A(i-1,2.2^^^’^‘^"^^  + A(i,2j-2)) 

< A(i-l,A(i,2j-l))  by  (5.1?) 

= A(i,2j)  for  i,  j>2,  if  B(i,  j-l)+l  - A(i,2j-2) 

and  B(i-1,2®^^’'^'"^^)+1  A(i-1, 2-2®^^’ ; 


Theorem  ^ ,3«  For  any  k,  s > 1 , let  T be  a complete  binary  tree  of 
heiglit  h > B(k,  s)  . Let  1 ^ ^ be  a set  of  pairwise 

unrelated  vertices  in  T , each  of  height  strictly  less  than  h-B(k, s)  , 
such  that  exactly  s vertices  in  (v^j  occur  in  each  subtree  of  T 
rooted  at  a vertex  of  height  h-B(k,  s)  . Then  for  n = 2'”  -1  and 

(^^  ^-VlAWA  nr'  m r'  m4‘  HVr  n mm  -m  mmln  T mm  -P  mm  n An  n min 


m = s2 


there  is  a set  union  problem  for  which 


(5.21)  the  union  tree  is  T ; 

(5.22)  the  set  of  finds  is  f find(Vj^ ) | 1 < i < m]  ; 

(5.23)  the  answer  to  each  find  is  a vertex  of  height  strictly  greater 
than  h-B(k, s)  ; and 

(5. 24)  any  link  solution  has  length  at  least  km  , even  if  every  edge 

(v,  w)  such  that  v - w and  h(v)  < h-B(k,  s)  in  T is  allowed 

for  free,  and  after  each  link(v,  w)  every  edge  (x, y)  such 
* + * 

that  v-*x-.y-.w  is  added  for  free. 


Proof.  The  proof  is  by  double  induction  on  k and  s and  is  similar  to 
the  lower  bound  proof  in  [I7].  Bupj'ose  k = 1 . Consider  any  set  vinion 
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r ^ 

I 

i 

problem  consisting  of  n-1  unions  which  form  T followed  by  a find  on 
' • each  vertex  in  [v^]  . The  answer  to  each  find  is  the  root  of  T ; 

i (5.23)  holds  since  h > B(k,  s)  . None  of  the  originally  free  edges  solves 

> 

I a find.  Since  the  vertices  in  {v^}  are  pairwise  unrelated^  any  link(x,y) 

j can  solve  only  one  find,  even  including  the  appropriate  free  edges. 

i 

I Thus  (5.2U)  holds. 

( Suppose  the  theorem  holds  for  k-1  , s = 2 . The  following  argument 

proves  the  theorem  for  k with  s = 1 . Suppose  the  hypotheses  of  the 
theorem  hold.  Let  {u^  1 ^ 5 ^ 5 "the  set  of  vertices  of  height 

t ' + 

I h-B(k,  1)  in  T , numbered  so  that  u^^  -•  v^  . The  vertices  in  [u^} 

I 

i are  pairwise  unrelated  and  exactly,'  two  occur  in  each  subtree  of  T rooted  | 

I at  a vertex  of  height  h-B(k,  l)-»-l  = n-B(k-l,  2)  . By  the  induction  j 

I hypothesis  there  is  a set  union  problem  satisfying  the  theorem  for 

k'  = k-1  , s'  = 2 , T , [u^]  . Let  the  sequence  of  finds  and  unions  in  ! 

i this  set  union  problem  be  . Form  from  P^  by  replacing  each 

i find(Uj^ ) by  flnd(v^ ) . I claim  the  resulting  sequence  satisfies  the 

t theorem  for  k , s = 1 , T , [v^]  . 

I Certainly  (5.21)  - (5.23)  hold.  Consider  any  sequence  of  links 

I 

j which  carries  out  P^  , allowing  for  free  the  edges  described  in  (5.2^+). 

t 

j Form  a sequence  from  by  replacing  each  link(x, y)  such  that 

! X 

I -•  y for  some  (uniquely  determined)  i by  link(x, u^^)  . Delete  from 

j all  links  which  do  not  create  new  edges.  I claim  carries  out 

P^  (allowing  appropriate  edges  for  free)  and  that  js^^j  < Is^j-m  . 

The  following  property  is  true  initially  and  is  preserved  if  and 

I 

1 Sg  are  executed  in  parallel  (on  separate  graphs). 
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(5.25)  For  1 < i < m , is  adjacent  in  the  graph  manipulated  by 

to  all  vertices  adjacent  to  at  least  one  descendmit  of 
in  the  graph  manipulated  by  . 

It  follows  that  carries  out  . 

For  any  , consider  the  first  link(x,y)  in  such  that 

+ 

X — — y . There  must  be  such  a link  since  none  of  the  initially 

free  edges  solves  find(v^)  by  (5.23).  There  must  be  a path  of  length  two, 
say  (x,  z)(z,y)  , between  x and  y in  the  graph  existing  before  the  link. 

-K-  ^ 

Furthermore  z must  satisfy  u.  -•  z -«  v.  . It  follows  that  (x, u.  ) is 

1 1 \ / 

an  edge  of  the  existing  graph.  Thus  need  not  contain  an 

instruction  link(x, Uj^)  corresponding  to  link(x, y)  . This  is  true  for 
any  value  of  i . Hence  |s^[  < ISgj-m  . 

Since  (k-l)m  < js^j  by  the  induction  hypothesis,  [Sg]  km  , 
and  (5.2U)  holds. 

Suppose  the  theorem  holds  for  k , s-1  and  also  for  k-1  , B(k,  s-1)  . 

The  following  argument  proves  the  theorem  for  k , s . Suppose  the 

B ( k s ) 

hypotheses  of  the  theorem  hold.  Let  {w^  I 1 < i < 2 ^ ^ ] be  a subset 

of  [v^ } such  that  exactly  one  vertex  w^  occurs  in  each  subtree  of  T 

rooted  at  a vertex  of  height  h-B(k,  s)  . Let  [u^  1 ^ 1 ^ 5 be 

the  set  of  vertices  of  height  h-B(k,  s)  , n\mbered  so  that  ^ • 

Consider  the  subtrees  T.  , 1 < J 1 2®^^'^^  B(k, s-1)  ^ mooted  at 

J 

B (k  s-1) 

vertices  of  height  h-B(k,  s)  + B(k,  s-1)  = h-B(k-l,2  ' ’ '^ ) in  T . Each 

B fk  s-1 ) 

subtree  T.  contains  (s-l)2  ^ ^ ' vertices  in  [v. ]-{w. } , exactly 

J XI 

E-1  in  each  subtree  rooted  at  a vertex  of  height  h-B(k,  s)  . By  the 
induction  hypothesis  there  is  a set  union  problem  satisfying  the  theorem 
for  k’  = k , s’  = s-1  , T . , [v  ) v is  a vertex  in  T . and  v c {v. }- [w. } ] . 

Let  Pj  be  the  sequence  of  unions  and  finds  in  this  set  union  problem. 
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The  vertices  in  the  set  [u^]  are  pairwise  unrelated  and  exactly 
B(k  s-1) 

2 ' ^ occur  in  each  subtree  T.  of  T . By  the  induction  hypothesis 

t] 

there  is  a set  union  problem  satisfying  the  theorem  for  k'  = k-1  , 
s'  = 2 ^ ' , T , {u^}  . Let  Q be  the  sequence  of  unions  and  finds 

in  this  set  lanion  problem.  The  sequence  Q can  be  permuted,  without 
increasing  the  mrniber  of  links  required  to  carry  out  Q , so  that  all 
unions  forming  the  subtrees  T.  occur  before  a.1 1 other  operations. 

Let  Q'  be  foimied  from  the  permuted  version  of  Q by  deleting  all 
unions  forming  the  subtrees  T . > let  Q"  be  formed  from  Q'  by  replacing 
each  find(u^)  by  find(wj^)  , and  let  P"  = • • •>  f s )-B(k,  s-1)' ' 

I claim  F"  defines  a set  union  problem  ^ich  satisfies  the  theorem  for 
k , s , T , [v^]  . 


Certainly  (5.21)  - (5.25)  hold.  Consider  any  sequence  S"  of  links 
vAiich  carries  out  P"  , allowing  for  free  the  edges  described  in  (5.2b). 
Form  a new  sequence  S from  S"  by  replacing  each  link(x,y ) such  that 
w^  y for  some  (uniquely  determined)  i by  link(x,  u^)  . Delete  from 
S all  links  which  do  not  create  new  edges.  The  following  property  is 
true  initially  and  is  preserved  if  S and  S"  are  executed  in  parallel 
(on  separate  graphs). 


Bfk  ^ 

(5.26)  For  1 < i p 2 ' , u^  is  adjacent  in  the  graph  manipulated 

by  S to  all  vertices  adjacent  to  at  least  one  descendant  of  w^ 
in  the  graph  manipulated  by  S"  . 


It  follows  by  an  argument  like  that  in  the  pi-evious  case  that  S 


carries  out  P-  = V . . . , Q’  and  that 

\n\  < . C can  be  written  as  S = S^,Sg,  ...,S  s_l)' U , 
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allowing 


where  S.  carries  out  P-  for  1 < i < , 

for  free  the  edges  described  in  (5.24),  and  U carries  out  Q,'  , 

allowing  for  free  the  edges  (v, w)  such  that  v ^ w and 

h(v)  < h-B(k-l,  2^^^’ ^ and  after  each  link(v,  w)  allowing  for  free 

* + * 

the  edges  (x,y)  such  that  v -*  x y -*  w . This  means  that  U carries 
out  Q , allowing  the  appropriate  edges  for  free.  By  (5.24), 

|S.  1 > k(s-l)2®(^^"-^)  for  1 < i < 2B(k,s)-B(k,s-l)  ^ 

luj  2 (k-l)2^^^’^^  . It  follows  that 

1S"1  _>  lsl  + 2®^^'")  >k(s-l)2®(^'"U(k-l)2^(^'"U2B(^'")  = ks2®(^'")  = 

Thus  (5.24)  holds.  By  double  induction,  the  theorem  is  true  in  general. 

Corollary  5.1.  Let  k,  s > 1 . Let  T be  a complete  binary  tree  of 

height  B(k,  s)  . Then  there  is  a set  union  problem  vdiose  union  tree 

S ^ Ic  s ^ 

is  T , which  contains  m = s2  ' ’ finds,  and  -vAiich  requires  at  least 
(k-l)m  links  for  its  solution. 

I 

Proof.  Choose  ■?  > 1 such  that  2 > s . Let  T'  be  a complete  binary 
tree  formed  by  replacing  each  leaf  of  T by  a complete  binary  tree  of 
hei^t  t . Let  (v^  I ^ ^ vertices  satisfying  the 

hypotheses  of  Theorem  5.5  for  k , s , T'  . For  1 < i < m , let  u^ 
be  the  vertex  of  height  t in  T'  such  that  ^ ^ ' 4e 

a sequence  of  lonions  and  finds  defining  a set  union  problem  satisfying 
the  conclusions  of  Theorem  5.3  for  k , s , T'  , {v^}  . Without  loss  of 
generality  we  can  assume  that  the  unions  which  form  the  subtrees  of  T’ 
rooted  at  hei^t  I occur  at  the  front  of  P'  . 

Form  P from  P'  by  deleting  the  unions  which  form  the  subtrees 
of  T’  rooted  at  height  I and  replacing  each  find(v^)  by  find(Uj^)  . 


We  claim  P defines  a set  union  problem  satisfying  the  conclusions  of  the 


corollary.  Certainly  P contains  m finds  and  the  xmion  tree  of  F 


is  T . Suppose  S is  a sequence  of  links  -which  carries  out  P . 

Form  S'  from  S by  following  each  link(x,Uj^)  viiich  solves  a 
find(u^)  by  link(x,Vj^)  . Then  S'  carries  out  P*  if  all  ed^es 
(v,  w)  with  h(v)  < i are  allowed  for  free.  Thus  ]S'  ) > km  , and 
jS"!  > (k-l)m  . □ 

Theorem  5.2,  Lemma  5-1,  and  Corollary  5.1  combine  to  establish 
the  main  result  of  this  paper. 

Theorem  5»^«  There  is  a positive  constant  c such  that,  for  aiJ. 
m > n > 1 , there  is  a set  union  problem  consisting  of  m finds  and 
n-1  intermixed  imions  whose  solution  by  reference  machine  requires  at 
least  cma(m,  n)  steps. 

Proof.  Let  s = • Choose  k as  large  as  possible  such  that 

gB(k, s)+l_^  < n . Partition  the  n elements  into  as  many  sets  as 

possible  of  size  2®^^’ ^^''’^-1  , plus  leftover  elements.  At  most  n/2 

B ^ Ic  s 1 ^ IL 

elements  are  left  over.  On  each  set  of  2 ’ -1  elements,  define 

a set  union  problem  satisfying  Corollary  5.1.  Concatenate  these  problems, 
add  enough  additional  unions  to  combine  all  elements,  including  the 
leftovers,  into  a single  set,  and  add  enough  additional  finds  to  bring 
the  total  to  m . 

The  resulting  set  union  problem  contains  m finds,  n-1  intermixed 
unions,  and  requires  at  least  (k-l)s2^^^^ ^ = (k-l)sn/U  > 
(k-l)m/8  links  for  its  solution.  By  Theorem  5.2,  this  set  union  problem 
requires  at  least  (k-l)m/32  - 5m/J+  - n > (k-75)m/32  reference  machine 
steps  for  its  solution. 


If  a(m,n)  2 , k > a(m,n)-l  in  this  construction  since 

B(a(m,n)-1,  s)+l  < A(o:(m, n)-l,2s)  by  Lemma  5.1 

< logg  n by  the  definition  of  a . 

Thus  the  selected  set  union  problem  requires  at  least 

(o;(m, n)-7^)m/52  > a(m,n)m/6k  reference  machine  steps,  if 

a(m,n)  > ll8  . But  if  a(m,n)  < lb8  , any  set  union  problem  requires 

at  least  m > ma(m, n)/lU8  reference  machine  steps.  Choosing 

c = 1/118  gives  the  theorem,  □ 
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Conclusions 


This  paper  has  described  a machine  model,  called  a reference  machine, 
suitable  for  analyzing  list  processing  problems.  The  model  is  similar  to 
several  previously  proposed  [ 8, 11, IP,  1' ] . Reference  machines  are  quite 
powerful;  Schlinhage  [l6]  has  shown  that  they  can  simulate  Turing  machines 
with  multidimensional  tapes  in  real  time,  and  one  can  show  that  they  can 
simulate  random  access  machines  with  logarithmic  cost  in  real  time. 

The  paper  has  analyzed  the  ability  of  reference  machines  to  compute 
disjoint  set  unions.  Under  certain  natural  restrictions,  all  reference 
machines  require  non-linear  time  to  solve  this  rroblera.  This  lower  bound 
characterizes  the  cfl'lciency  with  v/hlcli  one  can  represent  dynamic  inforr.atioi! 
of  a certain  kind  in  a list  stractui-e.  The  bound  does  not  require  that 
the  machine  be  deterrrJuistic,  or  that  the  program  of  the  machine  be  i'ixed 
’,rtille  the  problem  size  grows,  or  that  the  complexity  of  memory  (number  of 
fields  per  record)  be  fixed  while  the  ] roblem  size  grows. 

This  generality  is  achievcii  hy  m-jling  the  assumption  that  the 
description  of  each  set  is  stored  s<q -irately  and  that  moving  the 
description  oV  a seL  req'.iires  enstont  time  per  element.  Without  these 
assumptions  the  lower  hound  is.  not  v,-lid.  I conjecture,  however,  that 
the  lower  bound  holds  ii'  the  scji.arate  storage  ascumpition  is  replaced  by 
an  acG'oirrplion  about  the  complexity  of  memory;  namely,  that  every  record 
contains  only  a fixed  nur'.bcr  fields  independent  of  the  problem  size. 

The  paper  has  prcsent«-d  a number  of  knowr;  set  union  algorithms  and  has 
shown  that  they  all  fit  into  the  reference  machine  model.  One  of  the  algorltlir.. 
achieves  the  lower  bouid  to  within  a const^int  factor.  Tliis  .algorithm 
requires  that  arithmetic  be  performed,  but  the  arithmetic  can  be  simulated 


using  list  processing  with  only  a constant  factor  loss  in  running  time. 


I believe  that  any  algorithm,  even  one  which  uses  address  arithmetic, 
requires  non-linear  time  to  solve  the  set  union  problem.  Proving  such 
a statement  seems  to  reqxiire  a better  understanding  of  random  access 
machines. 

The  set  vinion  problem  can  be  generalized  to  a problem  requiring 
evaluation  of  functions  defined  on  paths  in  trees.  The  techniques  used 
here  and  in  [ly]  lead  to  a non-linear  lower  bound  for  some  sjecial 
cases  of  this  generalized  problem  [20].  Certain  cases  of  the  problem  can 
be  solved  in  almost-linear  time  by  using  complicated  extensions  of  the  best 
set  union  algorithti  presented  here  [l8].  Whether  the  most  general  version 
of  the  function  evaluation  problem  can  be  solved  in  almost-linear  time 
is  unknown. 

‘1 

! 
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procediore  quick  find 


Sq  <-  s et  ( parent  ; 


yrocedure  slow  union; 
while  ^ 0 do 
begin 


save  <-  next(r^); 
parent (r^)  - r^; 
next(r^)  *- nert(r^); 
next(r^)  - r^; 
rg  •-  save 
end; 


Table  t.l.  Programs  for  find  and  union  using  the  quick  find 
data  structure. 
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j'l-ocedure  slow  weighted  unjon; 


begin 

i*'  slze(r-,  ) ^ size(r^)  then 


begin 

^(r^)  « get(r^); 


" ^2 5 


n_Z£(r^)  <-  size(rj^)  + size(r^) ; 


slow  union 


Table  1*.'^.  } rogram  for  weighted  union  heuristic  with  quick  find  data 

structure. 


procedure  quick  union; 
parent (r^)  »-  r^; 


procedure  slow  find; 
begin 
root  — r^^; 

while  parent  (root)  ^ p d.o  root  ♦-  parent  (root) ; 
Sq  •-  set  (root) 
end; 


quick  weighted  union; 
if  size(r^  ) < size(r„)  then 

■ ■ ' X — £Z 

begin 

^(r^)  « set(r^); 

parent(rj_)  - r^; 

size(r^)  <-  size(rj^)  + size(r^^); 

end 

else  begin 

parent  (r^)  - 

size(rj^)  - size(rj^)  + size(r^) 
end ; 


find  with  q^atti  comyrejsion; 

begin 
clow  j'ind; 


current  — rj^; 

while  I arent( current)  ^ do 
begin 


save  - parent (current ) ; 
parent(current ) ►-  root; 
current  - cave 
end  end; 


Table  l.t. 


I'rograjnc  for  weighted  union  and  j ath  corq  rerc  ’ on  heuidctic 
with  quick  union  data  structure. 


I 


Time 

Quick  find 

0(mn) 

[1  ] 

with  wei^-ted  imion 

0(m  log  n) 

[1  ] 

Quick  union 

0(mn) 

[ 1+] 

with  wei^ted  union 

0(m  log  n) 

[ U] 

with  path  compression 

0(m*max(l,  log(n^/m)/log(2m/n) ) ) 

[17] 

with  both  heuristics 

0(m  a(m,n) ) 

[17] 

Table  U.5.  Worst-case  running  times  of  set  union  algorithms. 


1+1 


Time 


Quick  find 

O(n^) 

f21] 

with  weighted  union 

0(n) 

[10] 

Quick  union 

O(n^) 

[21] 

with  weighted  union 

0(n) 

[10] 

with  x:ath  compression 

*> 

with  both  heuristics 

0(n) 

[10] 

Table  h.C,  Average  rimning  timec  of  set  un.i  on  filgorillmc 
il‘  m and  n are  projortional. 


(i' 


Figure  U.l.  Data  structure  for  quick  find  algorithm. 

Sets  are  A = (a,  b,  c,  d,  e}  , B = [f,  g, h,  i}  . 


